Dynamic Bayesian Networks for Transliteration Discovery and Generation

This project is involved with extraction and transliteration of entity names between languages that use different writing systems (or alphabets). Extraction involves the automatic identification of sequences in parallel/comparable corpora/text that can be considered as proper entity names. On the other hand, transliteration generation involves automatic transformation of a source language name to a target language name across different writing systems while ensuring that pronunciation is maintained during and after the process. Bilingual entity name recognition and / or transliteration are processes aimed at improving performance in various Natural Language Processing (NLP) applications including: Machine Translation (MT), Cross Language Information Retrieval (CLIR), and Cross Language Information Extraction (CLIE). As an example, in recognition, there has been a growing interest in recognizing variations for the same entity name across different languages (Hsu et al. in both the source language that uses the Cyrillic alphabet and target languages that use the Roman alphabet. The presence of variations for the same entity may lead to incomplete search results and cause communication problems in a larger community of language users (Hsu et al., 2007). In medication, identification of drug names that look similar has been Source in Russian that uses the Cyrillic alphabet Transliterations in languages that use the Roman alphabet Table 1.1 Example illustrating variations for the same name both its source language and languages to which it was transliterated (Source: NewsExplorer Website 1) 1 NewsExplorer website can be accessed via Mbale is a town on the slopes of Mountain Elgon in Eastern Uganda Мбале является городом на склонах горных Elgon в Восточной Уганде Table 1.2: Example illustrating translation of a phrase with an Out of Vocabulary word found to be very helpful in reducing drug prescription errors (Kondrak and Dorr, 2004). To stress the need for machine transliteration, table 1.2 shows a problem where a machine translation system (Google 2 translate engine) encounters a new entity name " Elgon " in English for which it can not translate in a target language (Russian). Such a situation arises because the translation system does not have this word in its translation dictionary or lexicon. Machine transliteration is one of the best approaches to help deal with Out Of Vocabulary (OOV) problems. This report proposes work in the framework of Dynamic Bayesian Networks (DBNs) in which various model spaces are investigated with the aim of improving entity name recognition and machine transliteration across different writing …